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1. INTRODUCTION 

In what follows all matrices are assumed to have real entries, and square matrices 
are always assumed to be symmetric unless stated otherwise. The support of a k x n 
matrix A = {aij) will be denoted below by 

supp(A) = G {1,...,A;} X {l,...,n} : a^j ^ O}. 

If A is an n X n matrix, we denote the decreasing rearrangement of its eigenvalues by 

Ai(A)^A2(A)^---^A„(A). 

M" will always be assumed to be equipped with the standard scalar product (■, ■). Given 
a vector t> G M" and i G {1, . . . , n}, we denote by Vi the ith coordinate of v. Thus for 
M, f G M" we have (m, v) = Y17=i '^i'^i- 

Our goal here is to describe the following theorem of Batson, Spielman and Sri- 
vastava |BSS] . and to explain some of its recently discovered geometric applications. 
We expect that there exist many more applications of this fundamental fact in matrix 
theory. 

Theorem 1.1. — For every e G (0, 1) there exists c{£) = 0(l/£:^) with the following 
properties. Let G = {gij) he an n x n matrix with nonnegative entries. Then there 
exists an n X n matrix H = {hij) with nonnegative entries that satisfies the following 
conditions: 

1. supp(i/) C supp(G'). 

2. The cardinality of the support of H satisfies |supp(if)| ^ c{e)n. 

3. For every x G M" we have 

n n n n n n 

i=l j=l 1=1 j=l i=l j=l 
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The second assertion of Theorem 11.11 is that the matrix H is sparse, yet due to the 
third assertion of Theorem II .li the quadratic form Yll^=i ^j=i 

nevertheless 

a good approximation of the quadratic form Yli=i ^i=i 9iji^i ~ Xj)^. For this reason 
Theorem 11.11 is called in the literature a sparsification theorem. 
The bound on |supp(if)| obtained in |BSSj is 

|supp(i7)| ^ 2 

Thus c(e) ^ 32/e^ + 0{l/e). There is no reason to expect that ([2j) is best possible, but 
a simple argument |BSSl Section 4] shows that necessarily c{e) ^ 

1.1. Historical discussion 

The sparsification problem that is solved (up to constant factors) by Theorem 11.11 
has been studied for some time in the theoretical computer science literature. The 
motivations for these investigations were algorithmic, and therefore there was emphasis 
on constructing the matrix H quickly. We will focus here on geometric applications 
of Theorem 11.11 for which the existential statement suffices, but we do wish to state 
that [BSSj shows that H can be constructed in time 0(r;,^|supp(G')|/£:^) = O^n^/e"^). 
For certain algorithmic applications this running time is too slow, and the literature 
contains works that yield weaker asymptotic bounds on |supp(iJ)| but have a faster 
construction time. While such tradeoffs are important variants of Theorem II. H they 
are not directly relevant to our discussion and we will not explain them here. For the 
applications described below, even a weaker bound of, say, |supp(if)| ^ c{e)n\ogn is 
insufficient. 

Benczur and Karger |BKj were the first to study the sparsification problem. They 
proved the existence of a matrix H with |supp(if)| ^ c(e)n log n, that satisfies the 
conclusion ([1]) only for Boolean vectors x G {0, 1}". In their series of works on fast 
solvers for certain linear systems |STlt IST2t IST3t IST4] , Spielman and Teng studied the 
sparsification problem as stated in Theorem II. ![ i.e., with the conclusion holding 
for every x G M". Specifically, in |ST4] . Spielman and Teng proved Theorem 11.11 with 
the weaker estimate |supp(if)| = O {n{\ogny /e"^). Spielman and Srivastava |SS1] im- 
proved this estimate on the size of the support of H to |supp(if)| = 0{n{\ogn) /e^). 
As we stated above. Theorem II. H which answers positively a conjecture of Spielman- 
Srivastava |SS1] . is due to Batson-Spielman-Srivastava |BSS] . who proved this sharp 
result via a new deterministic iterative technique (unlike the previous probabilistic ar- 
guments) that we will describe below. This beautiful new approach does not only yield 
an asymptotically sharp bound on |supp(i7)|: it gives for the first time a deterministic 
algorithm for constructing H (unlike the previous randomized algorithms), and it also 
gives additional results that will be described later. We refer to Srivastava's disserta- 
tion |Sr2j for a very nice and more complete exposition of these ideas. See also the work 
of Kolla-Makarychev-Saberi-Teng |KMSTj for additional results along these lines. 



n 



(2) 



1033-03 



1.2. Combinatorial interpretation 

Suppose that G is the adjacency matrix of the complete graph, i.e., the diago- 
nal entries of G vanish and gij = 1 if i 7^ J- Assume also that the matrix H of 
Theorem 11.11 happens to be a multiple of the adjacency matrix of a d- regular graph 
r = ({1, . . . ,n},E), i.e., for some 7 > and all i,j G {1, . . . ,n} we have hij = 7 if 
{i,j} G E and h^j = otherwise. Thus |supp(if)| = dn. By expanding the squares 
in ([T]) and some straightforward linear algebra, we see that ([T]) is equivalent to the bound 
{Xi{H)-XniH))/{Xi{H)-X2{H)) ^ 1 + e. Thus if e is small then the graph T is a good 
expander (see |HLWj for background on this topic). The Alon-Boppana bound [M] im- 
plies that H satisfies {Xi{H) - XniH)) / {Xi{H) - X2iH)) ^ l+4(l-o(l))\/d asn,d^ 00. 
This lower bound can be asymptotically attained since if F is a Ramanujan graph 
of Lubotzky-Phillips-Sarnak jLPS] then Xi{H) /j, XniH) /-f e [-2^/d^,2^/d^. 
Writing 1 + e = {d + 2^d-l) / {d - 2y/d-l) = 1 + 4(1 + o{l))/Vd, we see that the 
existence of Ramanujan graphs means that (in this special case of the complete graph) 
there exists a matrix H satisfying ([1]) with |supp(if)| = dn = 16?2(1 + o{l))/e'^. The 
bound on |supp(if)| in ([2]) shows that Thereom 11.11 achieves the optimal Ramanujan 
bound up to a factor of 2. For this reason Batson-Spielman-Srivastava call the matrices 
produced by Theorem ll.il "twice- Ramanujan sparsifiers" . Of course, this analogy is in- 
complete since while the matrix H is sparse, it need not be a multiple of the adjacency 
matrix of a graph, but rather an adjacency matrix of a weighted graph. Moreover, 
this graph has bounded average degree, rather than being a regular graph of bounded 
degree. Such weighted sparse (though non-regular) graphs still have useful pseudoran- 
dom properties (see [BSS[ Lemma 4.1]). Theorem II. II can be therefore viewed as a new 
deterministic construction of "expander-like" weighted graphs, with very good spectral 
gap. Moreover, it extends the notion of expander graphs since one can start with an 
arbitrary matrix G before applying the sparsification procedure, with the quality of the 
resulting expander (measured in terms of absolute spectral gap) being essentially the 
same as the quality of G as an expander. 

1.3. Structure of this paper. 

In Section [2] we state a stronger theorem (Theorem 12. ip of Batson-Spielman- 
Srivastava [BSS] . and prove that it implies Theorem II. 11 Section [3] contains the 
Batson-Spielman-Srivastava proof of this theorem, which is based on a highly orig- 
inal iterative argument. Section S] contains an application of Theorem 12. 1^ due to 
Srivastava [SrlJ, to approximate John decompositions. In section [S] we describe two ap- 
plications of Theorem 12. 1^ due to Newman- Rabinovich [NRj and Schechtman |Sche3] , 
to dimensionality reduction problems. Section [H] describes the work of Spielman- 
Srivastava |SS2j that shows how their proof technique for Theorem 12.11 can be used 
to prove a sharper version of the Bourgain-Tzafriri restricted invertibility principle. 
Section [7] contains concluding comments and some open problems. 
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2. A STRONGER THEOREM 



Batson-Spielman-Srivastava actually proved a stronger theorem that implies Theo- 
rem 11.11 The statement below is not identical to the statement in |BSS] , though it 
easily follows from it. This formulation is stated explicitly as Theorem 1.6 in Srivas- 
tava's dissertation |Sr2] . 

Theorem 2.1. — Fix e G (0, 1) and Tn,n E N. For every xi, . . . , Xm G IR" there exist 
si, . . . ,Sm G [0, oo) such that 

\{te{l,...,m}:s,^0}\ ^ ^ , (3) 

and for all y G M" we have 

mm m 
1=1 i=l i=l 

2.1. Deduction of Theorem 11.11 from Theorem 12.11 

Let G = (gij) be an n x n matrix with nonnegative entries. Note that the diagonal 
entries of G play no role in the conclusion of Theorem II. H so we may assume in what 
follows that Qii = for alH G {1, . . . , n}. 

The degree matrix associated to G is defined as usual by 



Ei=i 92j 



G 



\ 



En 
7=1 92,j 







v 







(5) 



E"=l dnj) 



and the Laplacian associated to G is defined by 

^ n n 

= Dg - G = - 'YY gijici - Cj) (g) {ci - Cj), (6) 
i=i j=i 

where ei, . . . ,e„ G is the standard basis of M". In the last equation in ([6]), and 
in what follows, we use standard tensor notation: for x,y G M" the linear operator 
X ® 2/ : — M" is given by (x ® y){z) = (x, z)y. 

Theorem 12. H applied to the vectors {y/gij (cj — Cj) : i, j E {1, . . . ,n} A i < j} C R"-, 
implies that there exist {sij : i,jE{l,...,n} A i < j} C [0, oo), at most \n/e'^~\ of 
which are nonzero, such that for every y G M" we have 

1=1 j=i+i ^ ^ 

Extend {sij)i^j to a symmetric matrix by setting su = and Sji = Sij if z > j, and 
define H = (hij) by hij = SijQij. Then supp(if) C supp(G') and |supp(if)| ^ 2 [n/e^]. 
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A straightforward computation shows that {AGy,y) = ^^i=i^^=i dijiVi — UjY and 
TJlZl Ej=i+i SijQij (ci - Cj, yf = \ hijiVi - Vj?- Thus, due to (HD Theo- 

rem [LT] follows, with the boimd on |supp(iJ)| as in ([2]). □ 



3. PROOF OF THEOREM 2.1 

Write A = Y^iLi^i ® ^i- Note that it suffices to prove Theorem 12.11 when A is the 
n X n identity matrix /. Indeed, by applying an arbitrarily small perturbation we may 
assume that A is invertible. If we then set yi = A~^l'^Xi then ^)^=\ Vi® Vi = and 
the conclusion of Theorem 12.11 for the vectors {yi, . . . ,?/m} implies the corresponding 
conclusion for the original vectors {xi, . . . , x^}. 

The situation is therefore as follows. We are given xi, . . . , x„ G satisfying 



Xi®Xi = I. (8) 



i=l 



Our goal is to find {sj}™ C [0, oo) such that at most \n/e ] of them are nonzero, and 



2 



For the ensuing argument it will be convenient to introduce the following notation: 

(10) 

The proof constructs by induction {tfcjfcli ^ [0, oo) and {yk}'kLi ^ {^^i; • • • 5 ^m} with 
the following properties. Setting = and Ai = J2]=i '^jVi'^Vj i G N, the following 
inequalities hold true: 

- - + z < A„(A,) ^ Ai(A,) <e(^- + ij, (11) 
and for every z G N we have 

n ^ n ^ 

^ ^ (H + ,) - A,(A,) = ^ (H + , _ 1) _ A^.(A,_0 ' ^^^^ 

and 

n n -j^ 

^ A,(A,) - (-1 + ,) ^ A,(A,.i) - (-f + z - 1) ■ ^^^^ 

(The sums in (fT2|) and (|T3|) represent the traces of certain matrices constructed from 
the Ai, and we will soon see that this is the source of their relevance.) 

If we continue this construction for k = f^/^^] steps, then by virtue of (11 II) we would 
have 



An(^A:) 7^ ~ 7 



2 
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By construction Ak = SiXi ® Xi with si, . . . , Sm € [0, oo) and at most k of them 
nonzero. Thus, this process would prove the desired inequahty ([9]). 

Note that while for our purposes we just need the spectral bounds in (ITT]) , we will 
need the additional conditions on the resolvent appearing in (fT2|) and (|T3l) in order 
for us to be able to perform the induction step. Note also that due to (ITT]) all the 
summands in (fT2|) and (fT3l) are positive. 

Suppose that i ^ 1 and we have already constructed the scalars ti, . . . , G [0, oo) 
and vectors . . . , G and let be the corresponding positive 

semidefinite matrix. The proof of Theorem 12.11 will be complete once we show that we 
can find ^ and yi G {xi, . . . , x^} so that the matrix Ai = Ai_i + tiUi ® Ui satisfies 
the conditions (fTTj). ([T^. (IT^. 

It follows from the inductive hypotheses ffTT]) and ffT5]l that 

1 ^y. 1 

n ^ 

— ^ 7 r = £<l. (14) 

Hence, since — is positive semidefinite, A„(A) ^ A„(Aj„i) > — ^ + i, implying 
the leftmost inequality in flTT]) . 

It will be convenient to introduce the following notation: 

a = Y—. ^ 7 r-^ 7 7>0, (15) 

+ ^ - 1) - x,iA-i) U Hi + ^) - >^AA^i) 

and 

n 2 n ^ 

^ = ^ X,iA-i) - (-7 + ~ S ^.(^-i) - (-7 + ^ - 1) ^ °' ^^^^ 

Note that f|T6l) makes sense since, as we have just seen, f|T4l) implies that we have 
A„(y4j_i) > + This, combined with ffTTl) . shows that the matrices 9 + i) I — Ai^i 
and Ai^i — (— ^ + i) I are positive definite, and hence also invertible. Therefore, for 
every j G {1, . . . , m} we can consider the following quantities: 



and 



= K (^^-^ " ("7 + ') ^) ' '''' ~ { (^^-^ " ("7 + ') ^) ' "'^^ ^ • ^^^^ 

The following lemma contains a crucial inequality between these quantities. 
Lemma 3.1. — We have Y.7=iPj > Y^=x'^r 
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Assuming Lemma [3.11 for the moment, we will show now how to complete the induc- 
tive construction. By Lemma 13.11 there exists j G {1, . . . ,m} for which Pj ^ aj. We 
will fix this j from now on. Denote 

ti = — - and Ui = Xj. (19) 

The following formula is straightforward to verify — it is known as the Sherman- 
Morrison formula (see [GV[ Section 2.1.3]): for every invertible n x n matrix A and 
every z G MJ^ we have 

{A + z®z)-' = A-' -^^ -A-\z®z)A-\ (20) 

1 + \A z, z) 

Note that ti {A^^lz ^ z)A^^) = {A~'^z,z). Hence, by taking the trace of the iden- 
tity (120|) we have 

tr {{A + z^ .)-) = tr (A-) - Y^^^^. (21) 
Now, for every t G (0, l/aj] we have 

E— = tr I (e (- + i] I - Ai^i - tXi (g) X,) ] 

+ -^Mi-i + t^j^^j) \\ \e J ' ) 



^ ^ (l + - + ^ - / (^ (I + ,) J _ A^_,Y' 



n /{e{^^+^)I-A.,)-\„x^ 
\ I \ L (22) 



inii \ ^ 1 

CSli 1 



]^^9{^ + ^-l)-XM^-ly ^^^^ 

In ([22]) we used the fact that t ^ 1/aj and aj > (^(^9 (^ + i) / — Aj_i) ^ Xj,Xj^. In 
particular, there is equality in ( 122|) ii t = 1/aj. As Ai = + -^Xj ® Xj, this 

proves f|T2|) . Inequality f l23|) also implies the rightmost inequality in ffTTjl . Indeed, 
assume for contradiction that Ai + -^Xj ® Xj^ ^ 6* (^ + i) . Since by the inductive 

hypothesis Ai(Aj_i) < (^ + z — l) < (^ + i), it follows by continuity that there 
exists t G (0, l/aj] for which Ai (Aj_i + txj (g) Xj) = 9 (^^ + i). This value of t would 
make Yl]=i 1/ (i" + ~ -^il^j-i + ''-^i ® ^j)) tie infinite, contradicting f l2^ since by 
the inductive hypothesis all the summands in the right-hand side of fl2^ are positive 
and finite. 
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It remains to prove (fT3l) — this is the only place where the condition fSj ^ aj will be 
used. We proceed as follows. 



Oil 



lO 1 



n 



tr y4i_i - ( \ I -\ Xj Xj 



a,- 



-1' 



-1 



X j , Xj 



Xj , X j 



-7 + «-l) 



This concludes the inductive construction, and hence also the proof of Theorem 12. 1|. 
provided of course that we prove the crucial inequality contained in Lemma [3. II 

Proof of Lemma \3.1\ — It is straightforward to check that the identity ([8]) implies that 
for every n x n matrix A we have 



=tr(A). 



Hence, 



E 



lfT71lA|[24ll 



+ )+- -a 



and, 



Eft 

Now, 



iITsIiaEH 



tr {{A-i - {-^ + ^) ly 



tr( (A^,- 



(24) 



(25) 



(26) 



tr( ie{- + i)l-A,_,"~' 



1 



n 

<E 

i 



^0(f+.)-A,(A,_i) 
1 



#V ^ = - (27) 
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and 



1 trf(e(^ + z)/-A_i 

m E?=i (g(f + ^)-A,(A.„0)'^ ^1 

OEU (f + ^) - ^.(^-i))"' (? + ^ - 1) - ^M^-i)y' " ^' 

2^«i ^ = (29) 

In order to use fl26l). we first bound b as follows. 



Hence, 



n ^ 



(A,-i(A._i) - (-f + z)) (A,_i(A_i) - (-f + ^ - 1)) 

1/2 



E 



1 



1/2 



V^(A,_,(A_O-H + 0)' , 



1/2 



(A,-i(A_0-(-7 + ^))' 
which simplifies to give the bound 



1" 1 _tr((A._i-H+^)/)-^) 



-> ^ = ^ 30 

Hence, 

Lemma [3.11 now follows from (12^ and (1311) . □ 

Remark 3.2. — In the inductive construction, instead of ensuring equality in f[T^ . we 
could have ensured equality in f[T^ and replaced the equality sign in (IT^ with the 
inequality sign ^. This would be achieved by choosing ti = in (|T9|) . Alternatively 
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we could have chosen ti to be any value in the interval [l//9j, l/«j], in which case both 
inductive conditions (fT2|) and (fT3|) would be with the inequality sign ^. 

4. APPROXIMATE JOHN DECOMPOSITIONS 

Let B2 C M" be the unit ball with respect to the standard Euclidean metric. Recall 
that an ellipsoid £ = TB2 C is an image of B2 under an invertible linear transfor- 
mation T : — )■ M". Let C M" be a centrally symmetric (i.e., K = —K) convex 
body. John's theorem jJo] states that among the ellipsoids that contain there exists 
a unique ellipsoid of minimal volume. This ellipsoid is called the John ellipsoid of K. 
If the John ellipsoid of K happens to be i?2 , the body K is said to be in John position. 
For any K there is a linear invertible transformation T : — )■ such that TK is in 
John position. The Banach-Mazur distance between two centrally symmetric convex 
bodies i^', L C M", denoted d-QM{K., L), is the infimum over those s > for which there 
exists a linear operator T : M" — satisfying K C TL C sK. 

John [Jo] proved that if K is in John position then there exist contact points 
Xi, . . . ,Xm e (OK) n (S-Bg ) and positive weights Ci, . . . , > such that 

m 

J]c,x, = 0, (32) 

i=l 

and 

m 

CiXi ® Xi = I. (33) 

i=l 

When conditions f l5^ and fl5^ are satisfied we say that {xi, Cj}™,^ form a John decom- 
position of the identity. It is hard to overstate the importance of John decompositions 
in analysis and geometry, and we will not attempt to discuss their applications here. 
Interested readers are referred to |Bal2] for a taste of this rich field. 

John proved that one can always take m ^ n{n + l)/2. This bound cannot be 
improved in general (see |PTj for an even stronger result of this type). However, if 
one allows an arbitrarily small perturbation of the body K, it is possible to reduce the 
number of contact points with the John ellipsoid to grow linearly in n. This sharp result 
is a consequence of the Batson-Spielman-Srivastava sparsification theorem 12.11 and it 
was proved by Srivastava in |Srl] . The precise formulation of Srivastava's theorem is 
as follows. 

Theorem 4.1. — If K ^ is a centrally symmetric convex body and e G (0, 1) then 
there exists a convex body L C M" with d-Q^{K,L) ^ 1 + e such that L has at most 
m = 0{n/e'^) contact points with its John ellipsoid. 

The problem of perturbing a convex body so as to reduce the size of its John decom- 
position was studied by Rudelson in |Rulj . where the bound m ^ C{e)n{\ogn)'^ was 
obtained via a randomized construction. In |Ru2j Rudelson announced an improved 
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bound of m ^ C(e)nlogn(loglogn)^ using a different probabilistic argument based on 
majorizing measures, and in |Ru3] Rudelson obtained the bound m = 0{e~^n\ogn), 
which was the best known bound prior to Srivastava's work. 

The key step in all of these proofs is to extract from (155]) an approximate 
John decomposition. This amounts to finding weights Si,...,Sm € [0, oo), such 
that not many of them are nonzero, and such that we have the operator norm 
bound \\I — J2iLi^i^i ^ ^- This is exactly what Theorem 12.11 achieves, with 
\{i G {1, . . . ,m} : Si ^ 0}| ^ c[e)n. Prior to the deterministic construction of Batson- 
Spielman-Strivastava |BSS] . such approximate John decompositions were constructed 
by Rudelson via a random selection argument, and a corresponding operator- valued 
concentration inequality. In particular, Rudelson's bound |Ru3] m = 0(£:^^n log n) 
uses an influential argument of Pisier. Such methods are important to a variety of 
apphcations (see |RVt ITr2] ). and in particular this is how Spielman-Srivastava |SS1] 
proved their earlier 0[e~'^n\o<gn) sparsification theorem. While yielding suboptimal 
results, this method is important since it has almost linear (randomized) running time. 
We refer to the recent work of Adamczak, Litvak, Pajor and Tomczak-Jaegermann for 
deeper investigations of randomized approximations of certain decompositions of the 
identity (under additional assumptions). 



Proof of Theorem\4-.l\ — Suppose that K is in John position, and let {a;j,Cj}"^]^ be 



the corresponding John-decomposition. Since ® {y/cixi) = /, we may use 

Theorem 12.11 to find Si, . . . , s^, ^ 0, with at most 0{n/e'^) of them nonzero, such that 
if we set A = ^™ SjCjXj ® Xi, then the matrices A — I and (1 + e/4)/ — A are positive 
semidefinite. Thus — /|| ^ e/4. 

The rest of the proof follows the argument in |RuH IRu2] . Write £ = A^^'^Bl^. Then 
since ||y4 — /|| ^ e/4 we have 

(l-i)£csjc(i + £)f. 

Denote Ui = Xi/\\A''^^^Xi\\2 G dS and define 

H = conv (^{±y,},^j[j (y^^)) ' 

where J = {i G {1, . . . , m} : Si ^ 0}. Then H is a. centrally symmetric convex body, and 
by a straightforward argument one checks (see |RulURu2] ) that C if C (l-\-2e)K. 

Set L = A-^/^H. Since K C we have (dH) n (dS) = {±yi}^eJ, and therefore 
(dL) n (<95^) = {±Zi}i^j, where Zi = A'^/'^yi. Writing at = ^\\A^/^Xi\\2, we have 

m 

aiZi IS) Zi + ^ ai{-Zi) ® {-Zi) = ^ SiCi{A~^^^Xi) (g) (A'^^'^Xi) 

i=l 

(J2s^CiX,®x^ A-'/^ = A-'/'AA-'/^ = I. 



m 
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Hence {±Zi,ai}i^j form a John decomposition of the identity consisting of contact 
points of L and B2 ^ L. By John's uniqueness theorem [Jo] it follows that B2 is the 
John ellipsoid of L. □ 

Remark 4-2. — Rudelson |Ru2t [Ru3] also studied approximate John decompositions 
for non-centrally symmetric convex bodies. He proved that Theorem 14.11 holds if K is 
not necessarily centrally symmetric, with m = 0{e~'^nlogn). Note that in the non- 
symmetric setting one needs to define the Banach-Mazur appropriately: dBM{K,L) 
is the infimum over those s > for which there exists f G M" and a linear operator 
T : — i- M" satisfying K + v ^ TL C s{K + v). Srivastava |Srl] . based on a refinement 
of the proof technique of Theorem 12.11 proved that if i^' C R" is a convex body and 
£ e (0, 1), then there exists a convex body L C M" with dBM{K,L) ^ \/5 + e such 
that L has at most m = 0{n/e^) contact points with its John ellipsoid. Thus, it is 
possible to get bounded perturbations with linearly many contact points with the John 
ellipsoid, but it remains open whether this is possible with 1 + e perturbations. The 
problem is how to ensure condition fl32|) for an approximate John decomposition using 
the Batson-Spielman-Srivastava technique — for symmetric bodies this is not a problem 
since we can take the reflections of the points in the approximate John decomposition. 



5. DIMENSIONALITY REDUCTION IN Lj, SPACES 

Fix p ^ 1. In what follows Lp denotes the space of p-integrable functions on [0, 1] 
(equipped with Lebesgue measure), and ip denotes the space M", equipped with the ip 
norm \\x\\p = {Y^^=i Since any n- dimensional subspace of L2 is isometric to £2, 

for any Xi, . . . , G L2 there exist yi, G £2 satisfying \\xi — Xj\\2 = WUi — yj\\2 for 

all i,j G {1, . . . ,n}. But, more is true if we allow errors: the Johnson-Lindenstrauss 
lemma [JL J says that for every xi, . . . , x„ G L2, e G (0, 1) there exists k = 0(e~^ logn) 
and Hi, . . . ,yn G £2 such that \\xi — Xj\\2 ^ \\yi — yj\\2 ^ (1 + £)\\xi — Xj\\2 for all 
i,j G {1, . . . , n}. This bound on k is known to be sharp up to a 0(log(l/e)) factor 

In Lp for p ^ 2 the situation is much more mysterious. Any n-points in Lp embed 
isometrically into £p for k = n{n — l)/2, and this bound on k is almost optimal [Ball] . 
If one is interested, as in the Johnson-Lindenstrauss lemma, in embeddings of ra-point 
subsets of Lp into £p with a 1 + e multiplicative error in the pairwise distances, then 
the best known bound on k, due to Schechtman |Sche2] . was 

( C{e)n\ogn pG[l,2), 
^ \ C{p,e)nP/Hogn pe (2, 00). ^ ' 

We will see now how Theorem 12.11 implies improvements to the bounds in f l34p when 
p = 1 and when p is an even integer. The bounds in ( 15^ for p ^ {1} U 2N remain the 
best currently known. We will start with the improvement when p = I, which is due to 
Newman and Rabinovich |NRj . In the case p G 2N, which is due to Schechtman |Sche3] . 
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more is true: the claimed bound on k holds for embeddings of any n-dimensional linear 
subspace of Lp into ip, and when stated this way (rather than for n-point subsets of 
Lp) it is sharp [BDGJNj. 

5.1. Finite subsets of Li 

It is known that a Johnson-Lindenstrauss type result cannot hold in Li: Brinkman 
and Charikar |BCj proved that for any D > 1 there exists arbitrarily large n-point 
subsets {xi, . . . , Xn} C Li with the property that if they embed with distortion D into 
i'^ then necessarily k ^ n^^^ , where c > is a universal constant. Here, and in what 
follows, a metric space {X, d) is said to embed with distortion D into a normed space Y 
if there exists / : X — > satisfying d{x, y) ^ — fiy) \\ ^ Dd{x, y) for all x,y E X. 

No nontrivial restrictions on bi-Lipschitz dimensionality reduction are known for finite 
subsets of Lp, p G (l,C)o) \ {2}. On the positive side, as stated in flM|) . Schechtman 
proved |Sche2] that any n-point subset of Li embeds with distortion 1 + e into for 
some k ^ C{e)nlogn. The following theorem of Newman and Rabinovich [NRj gets 
the first asymptotic improvement over Schechtman's 1987 bound, and is based on the 
Batson-Spielman-Srivastava theorem. 

Theorem 5.1. — For any e G (0, 1), any n-point subset of Li embeds with distortion 
1 + £ into l\ for some k = 0{n/e'^). 

Proof. — Let /i, ...,/„ G Li be distinct. By the cut-cone representation of Li metrics, 
there exists nonnegative weights {wE}EQ{i,...,n} such that for all z,j G {1,...,^.} we 
have 

11/^- /.111 = E WE\lE{^-lEm- (35) 
£;c{l,...,ri} 

See |DLj for a proof of (1551) (see also [Nat Section 3] for a quick proof). 

For every C {1, . . . , n} define xe = \/we '^{^e ^« ^ (^i' ■ ■ ■ is the standard 
basis of M"). By Theorem 12.11 there exists a subset a C 2^^''"'^^ with \a\ = O^n/e"^), 
and nonnegative weights {sE}Eea, such that for every y G M" we have 

BC{l,...,n} \i&E ) Ee<7 \i&E / EC{l,...,n} \ieE / 

Define Zi, . . . , z„ G M*^ by Zi = {sEWElE{i))Eea- ^or i,j G {l,...,n} apply to 
the vector y = Ci — Cj, noting that for all E C {1, . . . ,n}, for this vector y we have 

ll/i-/j||i^ ^i?|ls(0 - li?(j)l < X^ssWij|li5(i) - li5(j)| 

EC{l,...,n} Beer 

= \\z,-z,h^ (l + e) Y ^i.|li.(^)-li^(j)l^ll/.-/,lli- □ 

EC{l,...,n} 
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Remark 5.2. — Talagrand |Tal] proved that any n-dimensional linear subspace of Li 
embeds with distortion 1 + e into with k ^ C{e)n\ogn. This strengthens Schecht- 
man's bound in (l34l) for n-point subsets of Li, since it achieves a low dimensional 
embedding of their span. It would be very interesting to remove the logn term in 
Talagrand's theorem, as this would clearly be best possible. Note that n-point subsets 
of Li can conceivably be embedded into with k <^ n. Embedding into at least n 
dimensions (with any finite distortion) is a barrier whenever the embedding proceeds 
by actually embedding the span of the given n points. The Newman-Rabinovich argu- 
ment based on sparsification proceeds differently, and one might hope that it could be 
used to break the n dimensions barrier for n-point subsets of Li. This turns out to be 
possible: the forthcoming paper }ANNj shows that for any D > 1, any ra-point subset 
of Li embeds with distortion D into with k = 0{n/D). 

5.2. Finite dimensional subspaces of Lp for even p 

Given an n G N and e G (0, 1), what is the smallest /c G N such that any n-dimensional 
subspace of Lp linearly embeds with distortion 1 + e into Epl This problem has been 
studied extensively [skeTl [Sche2l iBLMl [TiTl [Til [JSTl [gZl [Z^ [J52] . the best known 
bound on k being as follows. 

C(p, e)^ logn (log logn) ^ p G (0, 1) |Zvj . 

C{e)n\ogn p = l |Talj . 

C(£)nlogn(loglogn)2 pG(l,2) jT^ . 

clp,e)nPl^\ogn p G (2, oo) jBLM] . 



k ^ 



In particular, Bourgain, Lindenstrauss and Milman |BLMj proved that if p G (2, oo) 
then one can take k ^ C(p, e)n^''^ logn. It was long known |BDGJN] . by considering 
subspaces of Lp that are almost isometric to ^^at necessarily k ^ c(p, e)n^/^. We 
will now show an elegant argument of Schechtman, based on Theorem 12. H that removes 
the log n factor when p is an even integer, thus obtaining the first known sharp results 
for some values of p 7^ 2. 

Theorem 5.3. — Assume that p > 2 is an even integer, n G N and e G (0, 1). Then 
any n-dimensional subspace X of Lp embeds with distortion 1 + e into £p for some 
k ^ {cn/pY^"^ / , where c is a universal constant. 

Proof. — By a standard argument (approximating a net in the sphere of X by simple 
functions), we may assume that X C £™ for some finite (huge) m G N. In what 
follows, when we use multiplicative notation for vectors in M™', we mean coordinate- 
wise products, i.e., for x,y E M*", write xy = {xiyi, . . . ,Xmym) and for r G N write 



Let ui, . . . ,Un be a basis of X. Consider the following subspace of M™: 

p- 
2. 



Y = span (^{ufy/^ ■■■uf^: £ G N, ji, . . . , je e {1, . . . ,n}, pi + ■ ■ ■ + p, 
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Then 

p/2 



, ,,,, fn+p/2-l\ fl{]nV 



Thinking of y as a (i-dimensional subspace of let vi,...,Vd be an orthonormal 
basis of Y . Define xi, . . . , Xm G F by = X]j=i('^i) j, where as usual ei, . . . , Cm 
is the standard coordinate basis of M™. Note that by definition (since is 
an orthonormal basis of y), for every y & Y and for every i G {1, . . . j""^} we have 
{xi,y) = {y,ei) = yi. By Theorem 12. H there exists a subset a C {l,...,m} with 
|cr| = 0{d/{pe)'^) ^ (cn/p)P/^/e^, and {sjjigo- ^ (0, oo), such that for all y e Y we have 

m m 
1=1 iGcr 1=1 

In particular, since by the definition of Y for every x G X we have x^/^ G Y, 

Thus X I— )■ (Sj^'''^Xj)jgo- maps X into C 4''"^^^'' and has distortion 1 + e. □ 



Remark 5.4- — The bound on k in Theorem l5.3l is sharp also in terms of the dependence 
on p. See |Sche3] for more information on this topic. 



6. THE RESTRICTED INVERTIBILITY PRINCIPLE 

In this section square matrices are no longer assumed to be symmetric. The ensuing 
discussion does not deal with a direct application of the statement of Theorem 12. H 
but rather with an application of the method that was introduced by Batson-Spielman- 
Srivastava to prove Theorem 12.11 

Bourgain and Tzafriri studied in |BTlt IBT2t IBT3] conditions on matrices which 
ensure that they have large "well invertible" sub-matrices, where well invertibility refers 
to control of the operator norm of the inverse. Other than addressing a fundamental 
question, such phenomena are very important to a variety of interesting applications 
that we will not survey here. 

To state the main results of Bourgain-Tzafriri, we need the following notation. For 
a C {1, . . . ,n} let R„ : M."- M."' be given by restricting the coordinates to a, i.e., 
Ra{J2^=iCii€-i) = J^iea^i^i (^^ usual, {ci}^^^ is the standard coordinate basis of M"). 
In matrix notation, given an operator T : — )■ M", the operator R„TR*^ -.W^^W^ 
corresponds to the a x a sub-matrix {{Tei,ej))ij^a-- The operator norm of T (as an 
operator from £2 to £3) "will be denoted below by ||T||, and the Hilbert-Schmidt norm 

of T will be denoted ||T||hs = \JYh=i YTj=i{Teu e^Y- 

The following theorem from [BTlt IBT3j is known as the Bourgain-Tzafriri restricted 
invertibility principle. 
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Theorem 6.1. — There exist universal constants c,K>0 such that for every n G N 
and every linear operator T : R" — )• the following assertions hold true: 

1. // ||Tej||2 = 1 for all i G {l,...,n} then there exists a subset a C {l,...,n} 
satisfying 

cn 



such that R„T*TRt is invertihle and 



\T\ 



\{R„T*TRiy^\\ ^ K 



(38) 



(39) 



2. // {Tei,ei) = 1 for all i G {1, . . . ,n} then for all e G (0, 1) there exists a subset 
o C {1, . . . , n} satisfying 



\o\ ^ 



ce^n 



\T\ 



(40) 



such that RcrT*TRl. is invertihle and 

{R^T*TRiy^\\ 



(41) 

The quadratic dependence on e in (HOj) cannot be improved |BHKW] . Observe 
that (139|) is equivalent to the following assertion: 

2 



* - > a- 

2 *eCT 



(42) 



We note that if T satisfies the assumption of the first assertion of Theorem 16.11 then 
T*T satisfies the assumption of the second assertion of Theorem 16. II Hence, the second 
assertion of Theorem 16.11 implies the first assertion of Theorem 16.11 with f l39|) replaced 
by \{R„T*TRIY^\ ^ fl+£)||Tf and (|38|) replaced by the condition |a| ^ ceVll^f ■ 
In |SS2] Spielman and Srivastava proved the following theorem: 



Theorem 6.2. — Suppose that xi, . . . , G M" \ {0} satisfy 

m 

= /. 

i=l 

Then for every linear T : M" — )■ M" and e G (0, 1) there exists a C {1, 



(43) 



, m} with 



^ Ik IIhs 



(44) 



and such that for all {ctijigcr C M ti;e have 

2 



^^^^ Qj^J- 'x ^ 



(1-^)11^1 



HS 



a?. 



(45) 
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Theorem 16.21 implies the Bourgain-Tzafriri restricted invertibihty principle. Indeed, 
take Xi = Ci and note that if either ||Tej||2 = 1 for alH G {1, . . . , n} or (Tci, Cj) = 1 for all 
i G {1, . . . ,n} then HTH^g ^ n. The idea to improve the Bourgain-Tzafriri theorem in 
terms of Hilbert-Schmidt estimates is due to Vershynin, who proved in |Ve] a statement 
similar to Theorem 16.21 (with asymptotically worse dependence on e). Among the tools 
used in Vershynin's argument is the Bourgain-Tzafriri restricted invertibihty theorem 
itself, but we will see how the iterative approach of Section [3] yields a self-contained and 
quite simple proof of Theorem l6.2[ This new approach of Spielman-Srivastava has other 
advantages. Over the years, there was interest [BTl] IBT3i ITrlj ICTj in improving the 
quantitative estimates in Theorem 16.11 (i.e., the bounds on c,K, and the dependence 
\a\ on e and ||T||), and Theorem 16.21 yields the best known bounds. Moreover, it is 
not obvious that the subset a of Theorem 16.11 can be found in polynomial time. A 
randomized algorithm achieving this was recently found by Tropp |Trl] . and the work 
of Spielman-Srivastava yields a determinstic algorithm which finds in polynomial time 
a subset a satisfying the assertions of Theorem 16.21 



Before proceeding to an exposition of the proof of Theorem 16.21 in |SS2j . we wish 
to note that another important result of Bourgain-Tzafriri |BTH IBT2j is the following 
theorem, which is easily seen to imply the second assertion of Theorem 16.11 with the 
conclusion f HT]) replaced by || (i?o-7'-RCT)~^ || ^ 1 + This theorem is important for 
certain applications, and it would be interesting if it could be proved using the Spielman- 
Srivastava method as well. 

Theorem 6.3. — There is a universal constant c > such that for every £ > and 
n E N if an operator T : — )■ M" satisfies {Tei,ei) = for all i G {1, . . . ,n} then 
there exists a subset cr C {1, . . . ,r;,} satisfying |cr| ^ ce'^n and \\R„TR*\\ ^ ^HT^H. 

6.1. Proof of Theorem 112] 

The conclusion f HSjl of Theorem 16.21 is equivalent to the requirement that the matrix 

A = Y^iTxi) ® (Txi) (46) 

has \a\ eigenvalues at least (1 — £:)^||T||yg/m. Indeed, if B is the \a\ x n matrix whose 
rows are {Txi}i(z„, then A = B*B. The eigenvalues of A are therefore the same as the 
eigenvalues of the \a\ x \a\ Gram matrix BB* = {{Txi,Txj))ij^(j. The assertion that 
all the eigenvalues of BB* are at least (1 — £^)^||7'||hs/''^ identical to 
Define 

k = ■ (47) 

We will construct inductively yQ,yi, . . . ,yk E M."- with the following properties. We set 
1/0 = and require that yi, . . . ,yk G {xi, . . . , Xm}- Moreover, if for i G {0, . . . , k} we 
write 

&. = ^(mi^s-^rir), (48) 
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then the matrix 

i 

A, = Y^iTy,) ® {Ty,) (49) 
i=o 

has k eigenvalues bigger than hi and all its other eigenvalues equal (this holds vacuously 
for i = 0). Note that this requirement implies in particular that yi, . . . ,yk are distinct. 
Finally, we require that for every i e {1, . . . ,k} we have 

m m 

^ ((A, - bjy' Tx,,Tx,) < J2 ((^-1 - b^-lIy' Tx„ Tx,) . (50) 
i=i i=i 

The matrix Ak will then have the form (1461) with |cr| = fc, and have k eigenvalues greater 
than (1 — £:)^||T||^g/m, as required. It remains therefore to show that for i G {1, . . . , fc} 
there exists a vector yi satisfying the desired properties, assuming that yo,yi, ■ ■ ■ ,yi-i 
have already been selected. 

Lemma 6.4. — Denote 

m m 

/X = ^ {{Ai_i - k_iiy^ Txj, Txj) - ^ {{Ai^i - bjy^ Txj,Txj) . (51) 
i=i i=i 

(Since hi G (0,6i_i), the matrix (v4j_i — hiiy^ makes sense in (1^ . ) Then 



J2 ((A-i - bjy' TT* (A-i - hjy^ Tx„ Tx,) 

m 

< -/ij^ (1 + ((A,_i - b,iy'Tx„Tx,)) . (52) 

Assuming the validity of Lemma [6.41 for the moment, we will show how to complete 
the inductive construction. By (!52|) there exists j G {1, . . . ,m} satisfying 

{{A^^ - b,iy' TT* (A,_i - bjy' Tx„Tx,) 

<-fi{l + {iAi_^ - kiy' Txj,Tx,)) . (53) 

Our inductive choice will be yi = Xj. 

The matrix {Ai_i — bi^iiy^ — (Aj_i — biiy^ is positive definite, since by the induc- 
tive hypothesis its eigenvalues are all of the form (A — &i-i)^^ — (A — &i)^^ for some 
A G M that satisfies A > > 6j or A = (and since for i ^ k we have bi > 0). Hence 
/z ^ 0. Since the left hand side of ( 153|) is nonnegative, it follows that 

1 + {{Ai^, - biiy' Txj, Txj) < 0. (54) 

Since Ai = Ai_i + (Txj) ® (Txj), it follows from (1^ that 

tr {{A, - bay') - tr ((A,_i - hiy') = --^Vr — \ '\ > o, (55) 



1 + {iAi_, - biiy' Txj,Txj) 
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where in the last inequahty of (155|) we used the fact that — is positive 

definite. At the same time, by the inductive hypothesis Ai(Aj_i), . . . , Aj_i(Aj_i) > 
and Ai(Aj_i) = ■ ■ ■ = An(A-i) = 0. Since Ai — Ai^i is a rank one positive semidefinite 
matrix, the eigenvahies of Ai and Ai^i interlace (see |Baht Section III.2]; this result 
goes back to |Wey| ), and therefore 

Xi{A,) ^ Xi{A_i) ^ X2{Ai) ^ A2(A-i) \-i{A-i) ^ \{Ai), (56) 

and 

Xi{A_,) = ■■■ = XniA^i) = A,+i(A,) = ■ ■ ■ = Xn{Ai) = 0. (57) 

Hence, 

tr {{A, - bj)-') - tr ((A,_i - bj)-') 

m i + 1 + V f ^ 1 \ ^ MA) 



Xi{Ai) -bi h ^ \Xj{Ai) - bi Xj{Ai^i) - bij bi{Xi{Ai) - hY 

implying that Aj(Aj) > bi. 

Therefore, in order to establish the inductive step, it remains to prove (150|) . To this 
end, note that due to (H3|l and f l2^ for every n x n matrix A we have 

m 

J2 {ATxj,Txj) = tr {T*AT) . (58) 

Hence f lHI]) is equivalent to the inequality 

tr (T* (A-i - h.jy' T) > tr (T* {A, - hi)-' T) . (59) 

Now, 

tr (T* {Ai - bj)-' T) - tr (T* (A,_i - bj)-' T) 

ip tr (r* (A,_i - bil)-' {{Txj) ® (Tx,)) (A_i - &,/)-^ r) 

1 + {{A_^-ba)-'Txj,Tx,) 
{{A^i - bJ)-'TT* (A,_i - bJ)-'Tx,,Tx,) 
1 + {{A_^-biI)-'Txj,Tx,) 

< /i = tr (T (y4i_i - bi^il) T) - tr (T (ylj_i - 6^/) T) . 

This proves (!59|) . so all that remains in order to prove Theorem 16.21 is to prove 
Lemma 16.41 

Proof of Lemma \6.4\ — Using (l58i) we see that our goal (152|) is equivalent to the fol- 
lowing inequality 

tr (T* (Ai_i - bil)-' TT* (A-i - bil)-' T) 

<-li{m + tr (T* ( - 6,/) T) ) . (60) 
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Note that 

tr (T* - hiy' TT* ( A-i - bjy' T) 

^ \\T\\hT{{A^,-bJ)-'TT*{A^,-bJ)-') = \\T\\hi{T*{A,,-bjy^T). (61) 
The inductive hypothesis fl50|) . or its equivalent form f l59|) . imphes that 

tr (T* - k^jy' T) < tr (T* (Aq - 60/)"' T) 

= -ltr(T*T) = ^ # (62) 

Hence, 

tr (T* (A,_i - hiy' T)^<^ - (63) 

From (IM]) and (jSSD we see that in order to prove (pi it suffices to estabhsh the following 
inequality: 

||Tftr(r*(A,_i-6,J)-2T) + (54) 

To prove (164|) we ffist make some preparatory remarks. For r G {0, . . . , z — 1} let 
be the orthogonal projection on the image of and let Qr = I — Pr^Q the orthogonal 
projection on the kernel of Ar. Since = we have Qq = I. Moreover, because 
Ar = Aj—i + (Tyr) ® (Ti/r) and Aj—i, (Tyr) ® (Tyr) are both positive semidefinite, it 
follows that Ker(Ar) = Ker(y4r-i) fl (Txr)^- Therefore 

tr(g^_i - Qr) = dim(Ker(A^_i)) - dim(Ker(^)) ^ 1. (65) 

Hence, 

||Q,T||^g = tr (T*g,T) = ||g.-iT||^s - tr (T*(g,_i - Q,)T) 

ll65l l 

^ ||g,_ir||^s-||rftr(g,_i-g,) ^ ||g,_iT||^s - ||Tf . (66) 

Since go = /, fl66l) yields by induction the following useful bound: 

Next, since the nonzero eigenvalues of v4j_i are greater than the matrix 

T*Pj_i ((v4j-i — bi^il) (v4j_i — biiy"^^ Pi-iT is positive semidefinite. In particular, its 
trace is nonnegative, yielding the following estimate: 

^ tr (r*p,_i ((A,_i - 6,_i/) (A-i - hiy^) P.,^,T) 

which rearranges to the following inequality: 

(6,_i - 6,)tr (T*P,_i (A-i - 6./)"' P,:-iT) 

^ tr (T*P,_i (A-i - 6.-i/)"' P^-lT) - tr (r*P,_i (A-i - hiy^ P,.{r) . (68) 



1033-21 



Since - k^J) ^Qi-i = --g^Qi-i and Qi^i{Ai^i - hj) ^Qi-i = -^Qj-i, 

/i = tr(T*(P,_i + gi_i)(A,_i-6,_i/)-^(P,_i + g,_i)T) 
-tr (T*(P,_i + g,_i) (A,_i - 6,/)-^ (P,_i + Q,_i)T) 
= tr (T*P,_i - 6.-1/)"' P._iT) - tr (T*P,_i - bj)-' P,_iT) 

Gil o h- ^ —h- 

^ - 6,)tr (T*P,_i - bJ)-'P,^,T) + 7' , ' IIQ.-i^llls- (69) 

Also Qi-i{Ai_i - biiy^Qi^i = TiQi-i, and therefore 

i 

tr (T* - 6,/)-^ T) = tr (T*P._i - bjy' P,^{T) + 



6? 



1 



tr (r*i^_i - b,i)-'p,^{r) + ^lig.-iT 



IHS 



bi-i-bi bi \bi bi^i 



(i-£)||rp ■ h \h h-i, 

It follows that in order to prove the desired inequality flM|) . it suffices to show that the 
following inequality holds true: 



Since bi-i > bi and T*Pi_i (A-i — bil)'"^ Pi^iT is positive semidefinite, a consequence 
of f l69|l is that /i ^ ||gj_iT||Hg — ^-'-j-j- Hence, in order to prove flTOj) it suffices to 
show that 



6i v^* ^.-1/ ^.-1 

or equivalently, 

iiQ.-iTiiL ^ ^ ^riiHs - - mnw 

Oi-l — Oi 

which is a consequence of inequality f l67|) . that we proved earlier. □ 



7. NONLINEAR NOTIONS OF SPARSIFICATION 

Quadratic forms such as Yll=i X]j=i Qij ~ ^j)^ expressed in terms of the mutual 
distances between the points {xi, . . . , x„} C R. This feature makes them very useful for 
a variety of applications in metric geometry, where the Euclidean distance is replaced 
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by other geometries. We refer to |MNt INS] for a (partial) discussion of such issues. It 
would be useful to study the sparsification problem of Theorem 1 1.1 1 in the non- Euclidean 
setting as well, although the spectral arguments used by Batson-Spielman-Srivastava 
seem inadequate for addressing such nonlinear questions. 

In greatest generality one might consider an abstract set X, and a symmetric function 
(kernel) K : X x X [0, oo). Given an n x matrix G = (Qij), the goal would be to 
find a sparse n x n matrix H = (hij) satisfying 

n n n n n n 

^^9ijK{xi,Xj) ^ ^'^hijK{xi,Xj) ^ C^^gijK{xi,Xj), (71) 

i=l j=l i=l j=l i=l j=l 

for some constant C > and all Xi, . . . , x„ G X. 

Cases of geometric interest in ( FTTj) are when K{x,y) = d{x,yy, where d{-,-) is a 
metric on X and p > 0. When p ^ 2 even the case of the real line with the standard 
metric is unclear. Say that an n x n matrix H = (hij) is a p-sparsifier with quality C 
of an X n matrix G = (gij) if supp(if ) C supp(G) and there exists a scaling factor 
A > such that for every xi, . . . , a;„ G M we have 

n n n n n n 

X] XI - 1^ ^ X] S 1^* ~ 1^ ^ XI XI - Xj (72) 

i=l j=l i=l j=l i=l j=l 

By integrating (1721) we see that it is equivalent to the requirement that for every 
fi, . . . , fn ^ Lp we have 

n n n n n n 

^Y.Y.9^M - m ^Y.Y.hM - fx ^ cxY,Y.9^M - m- (73) 

i=l j=l i=l j=l 1=1 j=l 

By a classical theorem of Schoenberg |Scho] (see also |WWj ). if g ^ p then the metric 
space (M, \x — y]'^'^^) admits an isometric embedding into L2, which in turn is isometric 
to a subspace of Lp. It therefore follows from f l73|) that if if is a p-sparsifier of G with 
quality C then it is also a g-sparsifier of G with quality G for every q ^ p. In particular, 
when p G (0, 2), Theorem 11.11 implies that for every G a p-sparsifier H of quality 1 + e 
always exists with |supp(if)| = Oin/e^). 

When p > 2 it is open whether every matrix G has a good p-sparsifier H. By "good" 
we mean that the quality of the sparsifier H is small, and that |supp(/i)| is small. In 
particular, we ask whether every matrix G admits a p-sparisfiers H with quality 0^(1) 
(maybe even 1 + e) and |supp(if)| growing linearly with n. 

It was shown to us by Bo'az Klartag that if G = {gigj) is a product matrix with non- 
negative entries then Matousek's extrapolation argument for Poincare inequalities [Maj 
(see also [NSl Lemma 4.4]) can be used to show that if g > p and if is a p-sparsifier of 
G with quality C, then H is also a g-sparsifier of G with quality G'{G,p, q). However, 
we shall now present a simple example showing that a p-sparsifier of G need not be 
a g-sparsifier of G with quality independent of n for any q > p, for some matrix G 
(which is, of course, not a product matrix). This raises the question whether or not the 
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method of Batson-Spielman-Srivastava, i.e., Theorem II. ![ produces a matrix H which 
is a 0(l)-quahty p-sparsifier of G for some p > 2. 

Fix q > p, e > and n G N. Let G = (gij) be the n x n adjacency matrix of the 
weighted n-cycle, where one edge has weight 1, and all remaining edges have weight 
{n - iy~^/e, i.e., gin = Qni = 1, 

_ {n- 

912 — 921 — 923 — 932 — ■ ■ ■ — 9n-l,n — 9n,n-l — , 

e 

and all the other entries of G vanish. Let H = (hij) be the adjacency matrix of the 
same weighted graph, with the edge {l,n} deleted, i.e., hin = hni = and all the other 
entries of H coincide with the entries of G. It is immediate from the definition that 

^21=1 YTj=i 9ij\xi - Xj\P ^ Xir=i I^j=i hijl^i ~ all xi, . . . ,Xn e R. The reverse 

inequality is proved as follows: 

n n 

^ ^ ^ ^ 9ij I Xi Xj 1^ 2 |xx I'' ~l~ ~ ^ ^ \Xi | 

i=l j=l 




^ ^ I Xi Xi^i 1^ 



ly-i 



i=l 

n-1 

^ ^ \Xi 



i=l 



i=i j=i 



ij I Xi Xj I 



Hence if is a p-sparsifier of G with quality 1 + e. 

For the points Xi = i we have Yl^=i Yl]=i 9ij\^i ~ = 2(n — 1)'^ + 2{n — 1)^/^, and 
Sr=i X]j=i ^jjki ~ = 2(r;, — ly /e. At the same time, if ?/2 = 1 and Hi = for all 
i G {1, . . . , ra} \ {2}, we have ^"^^ ^J^^ - yj\i = ^"^^ ^J^^ h,j\yi - yj\i > 0. 
Thus, the quality of ii as a g-sparsifier of G is at least e{n — 1)"^"^, which tends to oo 
with n, since q > p. 



Acknowledgments. This paper is a survey of recent results of various authors, 
most notably Batson-Spielman-Srivastava |BSSj . Spielman-Srivastava |SSH ISS2j . Sri- 
vastava |SrH ISr2] . Newman- Rabinovich |NRj and Schechtman |Sche3] . Any differences 
between the presentation here and the results being surveyed are only cosmetic. I 
am very grateful to Alexandr Andoni, Tim Austin, Keith Ball, Bo'az Klartag, Ofer 
Neiman and especially Vincent Lafforgue, Gilles Pisier, Gideon Schechtman and Nikhil 
Srivastava, for helpful discussions and suggestions. 



1033-24 



REFERENCES 

[ALPT] R. Adamczak, A. E. Litvak, A. Pajor, N. Tomczak-Jaegermann - Quantita- 
tive estimates of the convergence of the empirical covariance matrix in log- concave 
ensembles, J. Amer. Math. Soc. 23 (2010), no. 2, 535-561. 

[Al] N. Alon - Problems and results in extremal combinatorics, Discrete Math. 273 
(2003), no. 1-3, 31-53. 

[ANN] A. Andoni, A. Naor, O. Neiman - On isomorphic dimension reduction in ii, 
preprint (2011). 

[Bah] R. Bahtia - Matrix analysis. Graduate Texts in Mathematics, 169. Springer- 
Verlag, New York, 1997. 

[Ball] K. M. Ball - Isometric embedding in Ip-spaces, European J. Combin. 11 (1990), 
no. 4, 305-311. 

[Bal2] K. M. Ball - An elementary introduction to modern convex geometry. In Flavors 
of Geometry, 1-58, Math. Sci. Res. Inst. Publ., 31, Cambridge Univ. Press, Cambridge, 
1997. 

[BSS] J. D. Batson, D. A. Spielman and N. Srivastava - Twice-Ramanujan sparsifiers. 
In Proceedings of the 41st Annual ACM Symposium on Theory of Computing (2009), 
255-262. Available at | http : //arxiv . org/abs/0808 . 0163] 

[BK] A. A. Bencziir and D. R. Karger - Approximatind s — t minimum cuts in 0{n'^) 
time. In Proceedings of the 28th Annual ACM Symposium on the Theory of Computing 
(1996), 47-55. 

[BDGJN] G. Bennet, L. E. Dor, V. Goodman, W. B. Johnson, C. M. Newman - On 
uncomplemented subspaces of Lp, 1 < p < 2, Israel J. Math. 26 (1977), no. 2, 178-187. 

[BHKW] K. Berman, H. Halpern, V. Kaftal, G. Weiss - Matrix norm inequalities and 
the relative Dixmier property. Integral Equations Operator Theory 11 (1988), no. 1, 
28-48. 

[BLM] J. Bourgain, J. Lindenstrauss, V. Milman - Approximation of zonoids by zono- 
topes. Acta Math. 162 (1989), no. 1-2, 73-141. 

[BTl] J. Bourgain and L. Tzafriri - Invertibility of large submatrices with applications 
to the geometry of Banach spaces and harmonic analysis, Israel J. Math. 57 (1987), 
137-224. 

[BT2] J. Bourgain and L. Tzafriri - Restricted invertibility of matrices and applica- 
tions. Analysis at Urbana, Vol. II (Urbana, IL, 1986-1987), 61-107, London Math. 
Soc. Lecture Note Ser., 138, Cambridge Univ. Press, Cambridge, 1989. 

[BT3] J. Bourgain and L. Tzafriri - On a problem of Kadison and Singer, J. Reine 
Angew. Math. 420 (1993), 1-43. 

[BC] B. Brinkman and M. Charikar - On the impossibility of dimension reduction in 
k, J. ACM 52 (2005), no. 5, 766-788. 



1033-25 



[CT] P. Casazza and J. Tremain - Revisiting the Bourgain-Tzafriri restricted invert- 
ibility theo- rem, Operators and Matrices 420 (2009), 97-110. 

[DL] M. M. Deza and M. Laurent - Geometry of cuts and metrics, Algorithms and 
Combinatorics, 15, Springer- Verlag, Berlin, 1997. 

[GV] G. H. Golub and C. F. Van Loan - Matrix computations. Third edition, Johns 
Hopkins Studies in the Mathematical Sciences. Johns Hopkins University Press, Bal- 
timore, MD, 1996. 

[HLW] S. Hoory, N. Linial, A. Wigderson - Expander graphs and their applications, 
Bull. Amer. Math. Soc. (N.S.) 43 (2006), no. 4, 439-561. 

[Jo] F. John - Extremum problems with inequalities as subsidiary conditions. Studies 
and Essays Presented to R. Courant on his 60th Birthday, January 8, 1948, 187-204. 
Interscience Publishers, Inc., New York, N. Y., 1948. 

[JL] W. B. Johnson and J. Lindenstrauss - Extensions of Lipschitz mappings into a 
Hubert space. Conference in modern analysis and probability (New Haven, Conn., 
1982), 189-206, Contemp. Math., 26, Amer. Math. Soc, Providence, RI, 1984. 

[JSl] W. B. Johnson and G. Schechtman - Remarks on Talagrand's deviation inequal- 
ity for Rademacher functions. Functional analysis (Austin, TX, 1987/1989), 72-77, 
Lecture Notes in Math., 1470, Springer, Berlin, 1991. 

[JS2] W. B. Johnson and G. Schechtman - Finite dimensional subspaces of Lp, Hand- 
book of the geometry of Banach spaces. Vol. I, 837-870, North-Holland, Amsterdam, 
2001. 

[KMST] A. Kolla, Y. Makarychev, A. Saberi and S-H. Teng - Subgraph sparsification 
and nearly optimal ultrasparsifiers. In Proceedings of the 42nd ACM Symposium on 



Theory of Computing (2010), 57-66. Available at |http : //arxiv . org/abs/0912 . 1623 



[LPS] A. Lubotzky, R. Phillips, P. Sarnak - Ramanujan graphs, Combinatorica 8 
(1988), no. 3, 261-277. 

[Ma] J. Matousek - On embedding expanders into lp spaces, Israel J. Math. 102 (1997), 
189-197. 

[MN] M. Mendel and A. Naor - Towards a calculus for non-linear spectral gaps. In Pro- 
ceedings of the Twenty-First Annual ACM-SIAM Symposium on Discrete Algorithms 
(2010), 236-255. Available at |http : //arxiv . org/abs/0910 . 2041 [ 

[Na] A. Naor - Li embeddings of the Heisenberg group and fast es- 
timation of graph isoperimetry, in Proceedings of the International 
Congress of Mathematicians, Hyderabad India, 2010. Available at 
[http: //www. cims .nyu. edu/~naor/h omepagef i les/ICM.pdf [ 

[NS] A. Naor and L. Silberman - Poincare inequalities, embeddings, and 
wild groups, preprint (2010), to appear in Compositio Math. Available at 



http : //www. cims .nyu. edu/~naor/homepagefiles/wilder.pdf, 



1033-26 



[NR] I. Newman and Y. Rabinovich - Finite volume spaces and sparsification, preprint 
(2010). Available at http : //arx iv . org/ abs/1002 . 3541, 

[Ni] A. Nilli - On the second eigenvalue of a graph, Discrete Math. 91 (1991), no. 2, 
207-210. 

[PT] A. Pelczynski and N. Tomczak-Jaegermann - On the length of faithful nuclear 
representations of finite rank operators, Mathematika 35 (1988), no. 1, 126-143. 

[Rul] M. Rudelson - Approximate John's decompositions. Geometric aspects of func- 
tional analysis (Israel, 1992-1994), 245-249, Oper. Theory Adv. Appl., 77, Birkhuser, 
Basel, 1995. 

[Ru2] M. Rudelson - Contact points of convex bodies, Israel J. Math. 101(1) (1997), 
92-124. 

[Ru3] M. Rudelson - Random vectors in the isotropic position, J. Funct. Anal. 163(1) 
(1999), 60-72. 

[RV] M. Rudelson and R. Vershynin - Sampling from large matrices: an approach 
through geometric functional analysis, J. ACM 54 (2007), no. 4, Art. 21, 19 pp. (elec- 
tronic). 

[Schel] G. Schechtman - Fine embeddings of finite-dimensional subspaces of Lp, 1 ^ 
p<2, into Proc. Amer. Math. Soc. 94 (1985), no. 4, 617-623. 

[Sche2] G. Schechtman - More on embedding subspaces of Lp in Compositio Math. 
61 (1987), no. 2, 159-169. 

[Sche3] G. Schechtman - Tight embedding of subspaces of Lp in for even p. Preprint 
(2010). Available at h ttp://arxiv.org/abs/1009. 106 1, 

[SZ] G. Schechtman and A. Zvavitch - Embedding subspaces of Lp into lp , < p < 1, 
Math. Nachr. 227 (2001), 133142. 

[Scho] I. J. Schoenberg - On certain metric spaces arising from Euclidean spaces by a 
change of metric and their imbedding in Hilbert space, Ann. of Math. (2) 38 (1937), 
no. 4, 787-793. 

[STl] D. A. Spielman and S-H. Teng - Nearly-linear time algorithms for graph parti- 
tioning, graph sparsification, and solving linear systems. In Proceedings of the 36th 
Annual ACM Symposium on Theory of Computing (2004), 81-90. 

[ST2] D. A. Spielman and S-H. Teng - Nearly-linear time algorithms for graph parti- 
tioning, graph sparsification, and solving linear systems. Preprint (2006). Available at 
http : //arxiv . or g/abs/ cs/0607105 

[ST3] D. A. Spielman and S-H. Teng - A local clustering algorithm for massive graphs 
and its application to nearly-linear time graph partitioning. Preprint (2008). Available 
at http : //arxiv . org/abs/0809 . 3232, 

[ST4] D. A. Spielman and S-H. Teng - Spectral sparsification of graphs. Preprint (2008). 
Available at ,http : //arxiv . org/abs/0808 . 4134, 



1033-27 



[SSI] D. A. Spielman and N. Srivastava - Graph sparsification by effective resistances, 
In Proceedings of the 40th Annual ACM Symposium on Theory of Computing (2008), 
563-568. Available at http://arxiy.org/abs/0803.0929, 

[SS2] D. A. Spielman and N. Srivastava - An elementary proof of the restricted 
invertibility theorem. Preprint (2009), to appear in Israel J. Math. Available at 
jhttp : //arxiv . org/abs/09 11.1114] 

Sri] N. Srivastava - On Contact points of convex bodies, Preprint (2009). Available at 



http : //www. cs . yale . edu/homes/srivastava/papers/contact .pdf , 

[Sr2] N. Srivastava - Spectral sparsification and restricted in- 
vertibility, Ph.D. thesis, Yale University (2010). Available at 
jhttp : / / www . cs ■ y ale . edu/homes/ srivastava/ dissertation . pdf , 

[Tal] M. Talagrand - Embedding subspaces of Li into l^ , Proc. Amer. Math. Soc. 108 
(1990), no. 2, 363-369. 

[Ta2] M. Talagrand - Embedding subspaces of Lp in 1^, Geometric aspects of functional 
analysis (Israel, 1992-1994), 311-325, Oper. Theory Adv. AppL, 77, Birkhuser, Basel, 
1995. 

[Trl] J. A. Tropp - Column subset selection, matrix factorization, and eigenvalue opti- 
mization. In Proceedings of the Nineteenth Annual ACM -SIAM Symposium on Dis- 
crete Algorithms (2009), 978-986. Available at h ttp : //arxiv . org/abs/0806 . 4404 , 

[Tr2] J. A. Tropp - User-friendly tail bounds for sums of random matrices, 2010. 
Preprint available at http : / / arxiv . org/ abs/ 1004 . 4389 [ 

[Ve] R. Vershynin - John's decompositions: selecting a large 

part, Israel J. Math. 122 (2001), 253-277. Available at 
'http: //www-personal .umich. edu/~romanv/papers/john. pdf , 

[WW] J. H. Wells and L. R. Williams - Embeddings and extensions in analysis, Ergeb- 
nisse der Mathematik und ihrer Grenzgebiete, Band 84. Springer- Verlag, New York- 
Heidelberg, 1975. 

[Wey] H. Weyl - Das asymptotische Verteilungsgesetz der Eigenwerte linearer par- 

tieller Differentialgleichungen (mit einer Anwendung auf die Theorie der Hohlraum- 

strahlung). Math. Ann. 71 (1912), no. 4, 441-479. 
[Zv] A. Zvavitch - More on embedding subspaces of Lp into lp , < p < 1, Geometric 

aspects of functional analysis, 269-280, Lecture Notes in Math., 1745, Springer, Berlin, 

2000. 



Assaf NAOR 

New York University 

Courant Institute 

251 Mercer street 

New York, NY 10012 - USA 

E-mail : naorScims . nyu . edu 



